Expressive Probability Models in Science

The paper is a brief summary of an invited talk given at the Discovery Science conference. The principal points are as follows: rst, that probability theory forms the basis for connecting hypotheses and data; second, that the expressive power of the probability models used in scientiic theory formation has expanded signiicantly; and nally, that still further expansion is required to tackle many problems of interest. This further expansion should combine probability theory with the expressive power of rst-order logical languages. The paper sketches an approximate inference method for representation systems of this kind. Classical philosophers of science have proposed a \deductive-nomological" approach in which observations are the logical consequence of hypotheses that explain them. In practice, of course, observations are subject to noise. In the simplest case, one attributes the noise to random perturbations within the measuring process. For example, the standard least-squares procedure for tting linear models to data eeectively assumes constant-variance Gaussian noise applied to each datum independently. In more complex situations, uncertainty enters the hypotheses themselves. In Mendelian genetics, for instance, characters are inherited through a random process; experimental data quantify, but do not eliminate, the uncertainty in the process. Probability theory provides the mathematical basis relating data to hypotheses when uncertainty is present. Given a set of data D, a set of possible hypotheses H, and a question X, the predicted answer according to the full Bayesian approach is P (XjD) = X H2H P (XjH)P(DjH)P(H) where is a normalizing constant, P (XjH) is the prediction of each hypothesis H, P (DjH) is the likelihood of the data given the hypothesis H and therefore incorporates the measurement process, and P (H) is the prior probability of H. Because H may be large, it is not always possible to calculate exact Bayesian predictions. Recently, Markov chain Monte Carlo (MCMC) methods have shown great promise for approximate Bayesian calculations 3], and will be discussed further below. In other cases, a single maximum a posteriori (MAP) hypothesis can be selected by maximizing P (DjH)P(H).